Monte Carlo Hypothesis Testing Research Articles

Clustering of Pediatric Brain Tumors in Texas, 2000-2017.

Risk factors for pediatric brain tumors are largely unknown. Identifying spatial clusters of these rare tumors on the basis of residential address may provide insights into childhood socio-environmental factors that increase susceptibility. From 2000-2017, the Texas Cancer Registry recorded 4305 primary brain tumors diagnosed among children (≤19 years old). We performed a spatial analysis in SaTScan to identify neighborhoods (census tracts) where the observed number of pediatric brain tumors was higher than expected. Within each census tract, the number of pediatric brain tumors was summed on the basis of residential address at diagnosis. The population estimate from the 2007-2011 American Community Survey of 0- to 19-year-olds was used as the at-risk population. p-values were calculated using Monte Carlo hypothesis testing. The age-standardized rate was 54.3 per 1,000,000. SaTScan identified twenty clusters, of which two were statistically significant (p < 0.05). Some of the clusters identified in Texas spatially implicated potential sources of environmental risk factors (e.g., proximity to petroleum production processes) to explore in future research. This work provides hypothesis-generating data for further investigations of spatially relevant risk factors of pediatric brain tumors in Texas.

A secure optimal placement strategy based on Monte Carlo simulation and hypothesis testing

The co-location attack evil pursues cloud computing until present. Our approach to address this challenge falls under the secure VM placement strategies which provide an a-priori knowledge about the security level. Besides, it gives the hand to control the maximum rate of the approved insecure placements. To do this, it uses the Monte Carlo simulation and hypothesis testing. We evaluate our approach for correctness in terms of security approximation and performance in terms of energy consumption. It outperforms the existing works in terms of energy consumption with a negligible time delay.

A model-backfeed deformation estimation method for revealing 20-year surface dynamics of the Groningen gas field using multi-platform SAR imagery

• The model-backfeed deformation estimation method offers more reliable InSAR deformation parameter estimations. • Multiple Hypothesis Testing is employed to determine the best deformation models. • Geolocation error ellipsoids and Monte Carlo methods are used for spatial integration. • Three SAR missions are used to map 20-year Groningen gas field subsidence. The Groningen gas field, the largest natural gas field in Europe, was discovered in 1959 and started its production in 1963. The Earth surface above it experienced subsidence over the past six decades because of gas extraction activities. To accurately reveal this surface movement with satellite SAR data, our study first proposes and demonstrates a model-backfeed (MBF) InSAR deformation estimation method to improve InSAR deformation time series modeling. This method allowed us to include a priori knowledge and to iteratively optimize functional and stochastic models. Using this method and employing a spatio-temporal SAR data integration method based upon Monte Carlo and Multiple Hypothesis Testing methods, we retrieved the 20-year subsidence history of the Groningen gas field by integrating 32 ERS-1/2, 68 Envisat and 82 Radarsat-2 SAR images. The results show that the maximum cumulative surface subsidence in this gas field has been as much as 15.5 cm between 1995 and 2015. In terms of precision and accuracy, our MBF method offered a better result than the standard Multi-Temporal InSAR analysis method: the Ensemble Coherence increased by 10%–19% and Spatio-Temporal Consistency decreased by 2%–20%. In terms of accuracy, our results better concur with the external GNSS reference observations. We further show that the spatio-temporal SAR data integration method has better links with multi-platform SAR data if the uncertainties of the InSAR geolocation and temporal deformation are included. The study demonstrates that the MBF method optimized the estimation of deformation parameters and mitigated unwrapping errors.

SPACE TIME CLUSTERS OF DENGUE FEVER IN MEDAN MUNICIPALITY, NORTH SUMATERA, INDONESIA

Dengue fever (DF) infection continues to present as a serious public health problem in Medan municipality, North Sumatera, Indonesia. The number of DF cases continuously increasing  recently. However, space time clusters of DF have not been investigated yet. A study was undertaken to detect clusters of DF incidence during 2015-2018 in Medan. Spatial geo-reference was conducted to 151 village coordinates by geocoding each village’s offices. A retrospective space-time scan statistic analysis based on population data and monthly DF incidence was  performed using SaTScan TM v9.4.4. Data of DF during 1 January 2015-31 December 2018 were analyzed using Poisson model to identify the villages at high risk of DF. The test of significance of the identified clusters of DF was based on comparing the likelihood ratio (LLR) against the null distribution obtained from Monte Carlo hypothesis testing. Total number of permutation was set to 999 and the significance level was set as 0.05. The highest LLR number was determined as the most likely cluster, while the rests were as the secondary clusters. This analysis identified 13 significant clusters. These DF clusters were initially spatially concentrated in the southwest and center of Medan and the last two years of study moved towards the northern part and identified in the last four months (September-December) of each year, which were the rainy seasons in the area. Most likely clusters were most frequently detected in the last three-year period of study in Anggrung village. Thirteen statistically significant DF clusters were identified in the 2015-2018 period. This may assist health authorities to improve the DF preventive strategies and develop public health interventions especially in the cluster’s area.

Visibility of the Gask Ridge road from simulated Watchtowers: A Monte Carlo testing approach

The Gask Ridge system is a series of forts, fortlets, and timber watchtowers situated along a Roman road in northern Scotland. The high intervisibility of the watchtowers in the Gask Ridge system has resulted in the proposal of two main functions: one that the watchtowers were a signalling system and two that the watchtowers provided visibility of the Gask Ridge road for surveillance and monitoring. Despite this, only the former function has been assessed. This paper explores the function of the watchtowers along the Gask Ridge road using computational methods, including Monte Carlo hypothesis testing. The analytical approach, which is documented and reproducible with accompanying code, rejects that the watchtowers were randomly located along the Gask Ridge road, instead favouring the alternative hypothesis that the watchtowers were located to maximise the visibility of the road. Furthermore, it is possible to claim that the need to monitor the road shows a causal relationship with the location of the watchtowers, rather than associative. The findings support the interpretation that the main function of the watchtowers was for the surveillance and monitoring of the Gask Ridge road, providing an early warning system of an attack from the Highlands (Woolliscroft, 1993).

The spatial clustering of dengue disease and risk susceptibility mapping: an approach towards sustainable health management in Kharagpur city, India

Dengue fever becomes one of the serious vector-borne diseases in the world, particularly in tropical countries. Mosquitoes borne disease transmission are mainly transpired by physical, environmental and socio-economic variables. Therefore, disease mapping is essential for monitoring, prediction, and prevention of dengue. Accordingly, the aims of this research are to find out the dengue outbreak, spatial pattern, understand the factors and construct a risk map to accurately predict and control of dengue. For this purpose, the Kharagpur city of West Bengal has been selected which is frequently encountered with the dengue outbreak. Epidemiological data were collected from the district health department. The spatial scan statistics of the Poisson model and Local Moran’s I is used for the mapping of spatial patterns and concentration of dengue and Monte Carlo hypothesis testing was applied to test the significance of the result. Therefore, multiple logistic regression analysis has been applied with selected parameters to produce a risk map for the entire study area. Finally, this study has proposed suitable site-specific management strategies. However, this study provides an example of disease risk estimation which will be remarkable for spatial analysis of epidemiological research as well as health planning and disease surveillance.

Truncated sequential Monte Carlo test with exact power

Monte Carlo hypothesis testing is extensively used for statistical inference. Surprisingly, despite the many theoretical advances in the field, statistical power performance of Monte Carlo tests remains an open question. Because the last assertion may sound questionable for some, the first goal in this paper is to show that the power performance of truncated Monte Carlo tests is still an unsolved question. The second goal here is to present a solution for this issue, that is, we introduce a truncated sequential Monte Carlo procedure with statistical power arbitrarily close to the power of the theoretical exact test. The most significant contribution of this work is the validity of our method for the general case of any test statistic.

An extension of Monte Carlo hypothesis tests

ABSTRACTThere are many hypothesis testing settings in which one can calculate a “reasonable” test statistic, but in which the null distribution of the statistic is unknown or completely intractable. Fortunately, in many such situations, it is possible to simulate values of the test statistic under the null hypothesis, in which case one can conduct a Monte Carlo test. A difficulty however arises in that Monte Carlo tests, as they are currently structured, are applicable only if ties cannot occur among the values of the test statistics. There is a frequently occurring scenario in which there are lots of ties, namely that in which the null distribution of the test statistic has a (single) point mass. It turns out that one can modify the current form of Monte Carlo tests so as to accommodate such settings. Developing this modification leads to an intriguing identity involving the binomial probability function and its derivatives. In this article, we will briefly explain the modified procedure, discuss simulation studies which demonstrate its efficacy, and provide a proof of the identity referred to above.

P-value approximations for spatial scan statistics using extreme value distributions.

Spatial scan statistics are widely applied to identify spatial clusters in geographic disease surveillance. To evaluate the statistical significance of detected clusters, Monte Carlo hypothesis testing is often used because the null distribution of spatial scan statistics is not known. A drawback of the method is that we have to increase the number of replications to obtain accurate p-values. Gumbel-based p-value approximations for spatial scan statistics have recently been proposed and evaluated for Poisson and Bernoulli models. In this study, we examine the use of a generalized extreme value distribution to approximate the null distribution of spatial scan statistics as well as the Gumbel distribution. Through simulation, p-value approximations using extreme value distributions for spatial scan statistics are assessed for multinomial and ordinal models in addition to Poisson and Bernoulli models.

Optimal generalized truncated sequential Monte Carlo test

When it is not possible to obtain the analytical null distribution of a test statistic U, Monte Carlo hypothesis tests can be used to perform the test. Monte Carlo tests are commonly used in a wide variety of applications, including spatial statistics, and biostatistics. Conventional Monte Carlo tests require the simulation of m independent copies from U under the null hypothesis, what is computationally intensive for large data sets. Truncated sequential Monte Carlo designs can be performed to reduce computational effort in such situations. Different truncated sequential procedures have been proposed. They work under restrictive assumptions on the distribution of U aiming to bound the power loss and to reduce execution time. Since the use of Monte Carlo tests are based on the situations where the null distribution of U is unknown, their results are not valid for the general case of any test statistic. In this paper, we derive an optimal scheme for truncated sequential Monte Carlo hypothesis tests. This scheme minimizes the expected number of simulations under any alternative hypothesis, and bounds the power loss in arbitrarily small values. The first advantage from this scheme is that the results concerning the power and the expected time are valid for any test statistic. Also, we present practical examples of optimal procedures for which the expected number of simulations are reduced by 60% in comparison with some of the best procedures in the literature.

Cluster recognition in spatial-temporal sequences: the case of forest fires

Forest fire sequences can be modelled as a stochastic point process where events are characterized by their spatial locations and occurrence in time. Cluster analysis permits the detection of the space/time pattern distribution of forest fires. These analyses are useful to assist fire-managers in identifying risk areas, implementing preventive measures and conducting strategies for an efficient distribution of the firefighting resources. This paper aims to identify hot spots in forest fire sequences by means of the space-time scan statistics permutation model (STSSP) and a geographical information system (GIS) for data and results visualization. The scan statistical methodology uses a scanning window, which moves across space and time, detecting local excesses of events in specific areas over a certain period of time. Finally, the statistical significance of each cluster is evaluated through Monte Carlo hypothesis testing. The case study is the forest fires registered by the Forest Service in Canton Ticino (Switzerland) from 1969 to 2008. This dataset consists of geo-referenced single events including the location of the ignition points and additional information. The data were aggregated into three sub-periods (considering important preventive legal dispositions) and two main ignition-causes (lightning and anthropogenic causes). Results revealed that forest fire events in Ticino are mainly clustered in the southern region where most of the population is settled. Our analysis uncovered local hot spots arising from extemporaneous arson activities. Results regarding the naturally-caused fires (lightning fires) disclosed two clusters detected in the northern mountainous area.

Bounding the resampling risk for sequential Monte Carlo implementation of hypothesis tests

Sequential designs can be used to save computation time in implementing Monte Carlo hypothesis tests. The motivation is to stop resampling if the early resamples provide enough information on the significance of the p-value of the original Monte Carlo test. In this paper, we consider a sequential design called the B-value design proposed by Lan and Wittes and construct the sequential design bounding the resampling risk, the probability that the accept/reject decision is different from the decision from complete enumeration. For the B-value design whose exact implementation can be done by using the algorithm proposed in Fay, Kim and Hachey, we first compare the expected resample size for different designs with comparable resampling risk. We show that the B-value design has considerable savings in expected resample size compared to a fixed resample or simple curtailed design, and comparable expected resample size to the iterative push out design of Fay and Follmann. The B-value design is more practical than the iterative push out design in that it is tractable even for small values of resampling risk, which was a challenge with the iterative push out design. We also propose an approximate B-value design that can be constructed without using a specially developed software and provides analytic insights on the choice of parameter values in constructing the exact B-value design.

Gumbel based p-value approximations for spatial scan statistics.

BackgroundThe spatial and space-time scan statistics are commonly applied for the detection of geographical disease clusters. Monte Carlo hypothesis testing is typically used to test whether the geographical clusters are statistically significant as there is no known way to calculate the null distribution analytically. In Monte Carlo hypothesis testing, simulated random data are generated multiple times under the null hypothesis, and the p-value is r/(R + 1), where R is the number of simulated random replicates of the data and r is the rank of the test statistic from the real data compared to the same test statistics calculated from each of the random data sets. A drawback to this powerful technique is that each additional digit of p-value precision requires ten times as many replicated datasets, and the additional processing can lead to excessive run times.ResultsWe propose a new method for obtaining more precise p-values with a given number of replicates. The collection of test statistics from the random replicates is used to estimate the true distribution of the test statistic under the null hypothesis by fitting a continuous distribution to these observations. The choice of distribution is critical, and for the spatial and space-time scan statistics, the extreme value Gumbel distribution performs very well while the gamma, normal and lognormal distributions perform poorly. From the fitted Gumbel distribution, we show that it is possible to estimate the analytical p-value with great precision even when the test statistic is far out in the tail beyond any of the test statistics observed in the simulated replicates. In addition, Gumbel-based rejection probabilities have smaller variability than Monte Carlo-based rejection probabilities, suggesting that the proposed approach may result in greater power than the true Monte Carlo hypothesis test for a given number of replicates.ConclusionsFor large data sets, it is often advantageous to replace computer intensive Monte Carlo hypothesis testing with this new method of fitting a Gumbel distribution to random data sets generated under the null, in order to reduce computation time and obtain much more precise p-values and slightly higher statistical power.

A generalized linear models approach to spatial scan statistics for covariate adjustment

The spatial scan statistic proposed by Kulldorff (Commun. Statist.-Theory Methods 1997; 26:1481-1496) is one of the most widely used methods for detecting spatial clusters and evaluating their statistical significance. However, it is not fully capable of adjusting for all types of confounding covariates. In this article, a generalized linear models (GLM) approach to construct spatial scan statistics, which is readily in a form for covariate adjustment, is proposed. Using GLM, spatial scan statistics for different probability models can be formulated in a single framework. The test statistic is based on the log-likelihood ratio test statistic and evaluated using Monte Carlo hypothesis testing. The proposed method is illustrated using Texas female breast cancer data concerning late versus early stage cancer cases with covariates of race/ethnicity and age group.

Spatial Clusters of Creutzfeldt-Jakob Disease Mortality in Japan between 1995 and 2004

Background: There is suggested to be a geographical difference in Creutzfeldt-Jakob disease (CJD) mortality in Japan. We performed a study to detect localized clusters and hot-spot areas of deaths from CJD in Japan during the 10-year period from 1995 to 2004. Methods: The diagnosis of CJD was taken from the death certificate (coded as A81.0 in the ICD-10). A total number of 1,168 CJD deaths (500 males and 668 females) were used for analysis using empirical Bayes estimates of standardized mortality ratios and the flexible spatial scan statistic to detect clusters. To detect the most likely cluster, p values were obtained using Monte Carlo hypothesis testing (with p < 0.05 as statistical significance).Results: The most likely cluster of CJD mortality was located in the northwest region from the base of Mt. Fuji, stretching over the two neighboring prefectures of Yamanashi and Shizuoka (relative risk = 2.28, p = 0.021). Some other clusters were detected but were not significant. Conclusions: The present study supports the evidence of geographical clustering of deaths from CJD at a specific location in Japan.

On Using Truncated Sequential Probability Ratio Test Boundaries for Monte Carlo Implementation of Hypothesis Tests

When designing programs or software for the implementation of Monte Carlo (MC) hypothesis tests, we can save computation time by using sequential stopping boundaries. Such boundaries imply stopping resampling after relatively few replications if the early replications indicate a very large or a very small p value. We study a truncated sequential probability ratio test (SPRT) boundary and provide a tractable algorithm to implement it. We review two properties desired of any MC p value, the validity of the p value and a small resampling risk, where resampling risk is the probability that the accept/reject decision will be different than the decision from complete enumeration. We show how the algorithm can be used to calculate a valid p value and confidence intervals for any truncated SPRT boundary. We show that a class of SPRT boundaries is minimax with respect to resampling risk and recommend a truncated version of boundaries in that class by comparing their resampling risk (RR) to the RR of fixed boundaries with the same maximum resample size. We study the lack of validity of some simple estimators of p values and offer a new, simple valid p value for the recommended truncated SPRT boundary. We explore the use of these methods in a practical example and provide the MChtest R package to perform the methods.

Runtime Verification for High-Confidence Systems: A Monte Carlo Approach

We present a new approach to runtime verification that utilizes classical statistical techniques such as Monte Carlo simulation, hypothesis testing, and confidence interval estimation. Our algorithm, MCM, uses sampling-policy automata to vary its sampling rate dynamically as a function of the current confidence it has in the correctness of the deployed system. We implemented MCM within the Aristotle tool environment, an extensible, GCC-based architecture for instrumenting C programs for the purpose of runtime monitoring. For a case study involving the dynamic allocation and deallocation of objects in the Linux kernel, our experimental results show that Aristotle reduces the runtime overhead due to monitoring, which is initially high when confidence is low, to levels low enough to be acceptable in the long term as confidence in the monitored system grows.

A spatial scan statistic for ordinal data

Spatial scan statistics are widely used for count data to detect geographical disease clusters of high or low incidence, mortality or prevalence and to evaluate their statistical significance. Some data are ordinal or continuous in nature, however, so that it is necessary to dichotomize the data to use a traditional scan statistic for count data. There is then a loss of information and the choice of cut-off point is often arbitrary. In this paper, we propose a spatial scan statistic for ordinal data, which allows us to analyse such data incorporating the ordinal structure without making any further assumptions. The test statistic is based on a likelihood ratio test and evaluated using Monte Carlo hypothesis testing. The proposed method is illustrated using prostate cancer grade and stage data from the Maryland Cancer Registry. The statistical power, sensitivity and positive predicted value of the test are examined through a simulation study.

Testing the model for one predator and two mutualistic prey species

With increasing frequency, models are being used to synthesize information and predict species response to ecosystem changes. Since it is reasonable to assume that models will continue to play a role in ecological and environmental studies, there is a need for a means of assessing model performance. The traditional approach of validation in terms of goodness-of-fit is such a process, but does not guarantee the “correctness” of model predictions. To increase confidence in the predictive qualities of the model, additional validation techniques are investigated, to ensure the effective and appropriate use of the model in the decision-making processes in managing wildlife undertakings. An initial graphical investigation is executed, to obtain a qualitative indication of model robustness, and is supplemented with quantitative techniques, such as model efficiency and Monte Carlo hypothesis testing, before conclusions are drawn with respect to model suitability.

Monte Carlo assessments of goodness-of-fit for ecological simulation models

One often develops stochastic ecologic simulation models based on local interactions between individuals or groups and bases systemic conclusions on trends summarized over multiple data sets generated from the model. In many cases, such models generate data sets (“realizations”) each violating the usual assumptions associated with traditional statistical tests of goodness-of-fit, most notably that of independent observations. Monte Carlo hypothesis tests applied to multiple realizations from such models provide appropriate goodness-of-fit tests regardless of within-model peculiarities. The Monte Carlo tests address the question “Do the observed data appear consistent with the model?” in contrast to the usual question “Does the model appear consistent with the observed data?”. In addition, such tests can make use of the same data sets used to draw systemic inference (i.e. the tests require no additional simulation runs). We illustrate the concept using Pearson’s chi-square statistic with correlated data. We also consider the behavior of a similar statistic and of “modeling efficiency” in assessing the fit of a simulation model for the spatial spread of raccoon rabies in Connecticut.